Adaptive Robotics - Final Report Extending Q-Learning to Infinite Spaces

One of the drawbacks of standard reinforcement learning techniques is that they only operate when both the state and action spaces are finite. Q-learning is one such algorithm. We propose an extension of Q-learning to infinite state and action sets called CHAMPAGNE, using a simple “Local Expert” function approximation method. We then experimentally test the performance of the algorithm on several navigation tasks. The algorithm is able to successfully solve a T-maze in pyrobot after reasonable training. We present results from varying the input type, reinforcement delay, and maximum memory size for this algorithm.